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The  rendezvous  phase  is  one  of  the  most  important  phases  in  future  orbital  servicing 
missions.  To  ensure  a  safe  approach  to  a  non-cooperative  target  satellite,  a  guidance, 
navigation  and  control  system  which  uses  measurements  from  optical  sensors  like 
cameras  was  designed  and  developed.  During  ground-based  rendezvous,  stability  pro¬ 
blems  induced  by  delayed  position  measurements  can  be  compensated  by  using 
a  specially  adapted  navigation  filter.  Within  the  VIBANASS  (Vision  BAsed  NAvigation 
Sensor  System)  test  campaign,  hardware-in-the-loop  tests  on  the  terrestrial,  robotic  based 
facility  EPOS  2.0  were  performed  to  test  and  verify  the  developed  guidance,  navigation 
and  control  algorithms  using  real  sensor  measurements.  We  could  demonstrate  several 
safe  rendezvous  test  cases  in  a  closed  loop  mode  integrating  the  VIBANASS  camera  system 
and  the  developed  guidance,  navigation  and  control  system  to  a  dynamic  rendezvous 
simulation. 

©  2014  IAA.  Published  by  Elsevier  Ltd.  All  rights  reserved. 


1.  Introduction 

Technology  for  on-orbit  servicing  is  developed  and 
missions  demonstrating  orbital  servicing  capabilities  are 
currently  in  preparation  [1-4],  Applications  of  on-orbit 
servicing  are  life  time  extension  of  partly  damaged  satel¬ 
lites  and  de-orbiting  of  in-operative  satellites  at  the  end  of 
their  life  [5],  The  rendezvous  phase  is  one  of  the  most 
critical  parts  of  a  robotic  servicing  mission  since  a  safe 
approach  to  a  non-cooperative  target  satellite  must  be 
conducted. 

To  approach  a  target  satellite,  optical  sensors  such  as 
mono  and  stereo  cameras  can  be  used  for  relative  naviga¬ 
tion.  Raw  camera  data  is  processed  to  provide  a  measure¬ 
ment  of  the  relative  position  between  the  two  spacecrafts. 
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Alternatively,  an  operator  on  ground  can  be  involved  who 
marks  the  position  of  the  target  satellite  in  camera  images 
and  thus  supports  or  replaces  the  automatic  object 
recognition. 

Rendezvous  and  docking  via  teleoperation  can  be  done 
in  the  framework  of  a  geostationary  robotic  serving  mis¬ 
sion  like  OLEV  [3]  and  within  a  low  Earth  orbit  mission 
using  a  data  relay  satellite  [2],  A  critical  problem  arising 
from  the  teleoperation  concept  is  time  delay.  The  delay  can 
cause  stability  problems  if  the  guidance,  navigation  and 
control  (GNC)  system  is  not  specially  developed  to  handle 
measurements  with  delay.  Stability  problems  are  very 
critical  if  high  dynamics  are  involved.  For  example,  during 
the  ROKVISS  experiment  [6],  where  modern  telerobotic 
technology  was  demonstrated,  special  control  methods 
were  applied  to  compensate  for  time  delays.  Using  stan¬ 
dard  communication  architecture,  time  delays  of  2-5  s  are 
expected  for  future  geostationary  on-orbit  servicing  [3] 
and  even  up  to  10  s  are  expected  in  the  ConeXPress-OLEV 
study  [7],  About  7  s  of  delay  (round-trip  delay)  was  faced 
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during  the  ETS-VII  mission  (Japanese  Engineering  Test 
Satellite  VII)  where  a  bilateral  teleoperation  experiment 
via  a  data  relay  satellite  was  performed  [8], 

Before  an  on-orbit  servicing  mission  can  be  launched, 
intensive  test,  verification  and  validation  of  the  rendez¬ 
vous  sensors  and  of  the  GNC  system  are  necessary. 
Hardware-in-the-loop  (HiL)  simulations  are  suitable  to 
test  real  hardware  like  cameras  in  an  almost  realistic 
scenario  including  simulation  of  illumination  conditions 
and  of  the  satellites’  relative  motion  [9-11].  In  a  closed 
loop  rendezvous  simulation  the  full  control  loop  can  be 
tested  consisting  of  sensor,  processing  unit,  guidance, 
navigation,  control  and  simulation  of  actuators  and  the 
spacecrafts’  dynamical  relative  motion. 

There  are  several  approaches  for  rendezvous  to  a  non- 
cooperative  satellite  in  space.  In  [12],  the  authors  cope  with 
the  problem  of  recognizing  an  uncooperative  target  satellite. 
Pose  estimation  methods  for  rendezvous  and  docking  appli¬ 
cations  are  proposed  in  [13],  A  GNC  system  for  rendezvous  is 
presented  in  [14],  First  demonstrations  of  ground-based 
rendezvous  were  already  performed  in  the  extended  phase 
of  the  PRISMA  mission.  The  German  Aerospace  Center 
demonstrated  a  controlled  approach  from  30  km  to  3  km 
in  the  ARGON  (Advanced  Rendezvous  demonstration  using 
GPS  and  Optical  Navigation)  experiment  [15]. 

Different  ground  facilities  exist  for  testing  GNC  algo¬ 
rithms  and  simulating  rendezvous  and  docking  maneu¬ 
vers.  In  the  framework  of  an  Italian  project  called  STEPS 
(Systems  and  Technologies  for  Space  Exploration)  a  Ren¬ 
dezvous  and  Docking  simulator  based  on  an  air-floating 
test  bed  (2D  platform)  and  GNC  algorithms  are  developed 
[16],  Hardware-in-the-loop  tests  of  a  3D  LIDAR  (Light 
Detection  and  Ranging)  executed  at  the  Spacecraft 
Robotics  Engineering  and  Controls  Lab  of  the  U.S.  Naval 
Research  Laboratory  are  described  in  [17],  The  authors 
observed  also  time  delays  induced  by  the  sensor  hardware 
and  by  data  interfaces.  The  European  Proximity  Operations 
Simulator  (EPOS)  2.0  [18,9]  is  a  robotic  based  test  bed 
allowing  for  rendezvous  and  docking  simulation  and 
sensor  and  GNC  system  verification.  Several  hardware-in- 
the-loop  tests  without  simulating  time  delay  have  been 
conducted  [11],  EPOS  2.0  was  also  involved  as  test  facility 
in  the  VIBANASS  project  (Vision  BAsed  NAvigation  Sensor 
System).  In  this  project  an  optical  rendezvous  and  docking 
camera  system  was  developed  [19]  and  tests  and  verifica¬ 
tion  of  the  camera  system  were  conducted  at  the  simulator 
EPOS  2.0  in  an  open  loop  mode  for  verification  of  the 
sensor  system  and  the  image  processing  module  [20,21], 
An  overview  of  ground  verification  methods  and  facilities 
is  given  in  the  survey  article  [22], 

Development  of  a  GNC  system  which  can  handle  delayed 
measurements  and  test  of  the  entire  control  loop  during  an 
approach  under  realistic  conditions  are  still  open  research 
tasks.  In  particular,  a  safe  approach  has  to  be  tested  and 
demonstrated  using  real  sensor  data  within  a  hardware-in- 
the  loop  simulation.  The  stability  of  the  control  loop  in  the 
case  of  delayed  measurements  has  to  be  proved.  In  the  final 
VIBANASS  test  campaign,  a  secondary  goal  was  to  demon¬ 
strate  closed  loop  rendezvous  simulation  with  the  VIBANASS 
camera  system.  Existing  approaches  [11]  using  Kalman  filter 
techniques  had  to  be  consequently  modified  for  this  task. 


The  objective  of  this  paper  is  to  present  a  hardware-in- 
the-loop  simulation  for  testing  closed  loop  rendezvous 
processes  within  the  VIBANASS  test  campaign  at  EPOS.  The 
main  task  is  to  develop  a  guidance,  navigation  and  control 
system  resulting  in  a  stable  control  loop  even  in  the  case  of 
delayed  position  measurements. 

Throughout  the  paper,  a  closed  loop  simulation  is 
a  simulation  in  which  the  output  of  a  GNC  system  is  fed 
back  to  the  simulation  of  the  actuators  and  spacecrafts’ 
dynamics.  Thus,  the  trajectory  of  an  approach  is  not  pre¬ 
defined  but  computed  in  real-time.  The  trajectory  depends 
on  the  sensor  measurements,  the  guidance  values  and  the 
design  of  the  navigation  filter  and  of  the  controller.  Time 
delay  is  the  time  difference  between  the  capturing  of  the 
images  by  a  camera  and  the  time  when  the  measurements 
are  available  for  the  navigation  system.  The  active  satellite 
in  an  on-orbit  servicing  mission  is  called  servicer  or  chaser, 
the  passive  satellite  is  called  client  or  target. 

2.  Methods 

2.1.  Hardware-in-the-loop  simulator  EPOS  2.0 

EPOS,  the  European  Proximity  Operations  Simulator,  is 
a  hardware-in-the-loop  simulator  focusing  on  the  last 
phase  (25-0  m)  of  Rendezvous  and  Docking  (RvD)  man¬ 
euvers  involving  two  spacecraft.  The  simulation  concept  of 
EPOS  takes  advantage  of  both  physical  and  computer 
simulation:  on  one  hand,  satellite  dynamics  cannot  be 
simulated  physically  on  ground  for  any  practical  purposes, 
while  numerical  simulation  can  be  realized  easily  and 
leads  to  very  good  results.  On  the  other  hand,  realistic 
sensor  output  including  various  physical  effects  is  very 
difficult  to  simulate  with  sufficient  quality  by  pure 
numeric  software.  EPOS  allows  for  the  combination  of 
numerical  satellite  dynamics  simulation  and  real  RvD 
sensor  hardware. 

Fig.  1  illustrates  the  facility  layout.  The  central  elements 
constitute  two  standard  industrial  robots.  Robot  1  in  Fig.  1 
is  a  KUKA  KR100HA,  which  can  carry  a  maximum  payload 
of  100  kg.  It  is  mounted  on  a  linear  rail  with  a  length  of 
25  m.  The  robot  can  be  moved  on  that  rail  in  order  to 
simulate  the  relative  distance  of  two  satellites  in  space. 
The  other  robot  (robot  2)  is  a  KUKA  KR240.  Its  maximum 
payload  is  240  kg.  In  contrast  to  robot  1,  it  is  mounted  at 
the  end  of  the  rail,  its  base  fixed  in  the  laboratory.  Each 
robot  is  equipped  with  a  breadboard  attached  to  the  tool 
flange  which  can  be  used  to  mount  satellite  mockups  or 
sensor  devices. 

If  visual  RvD  sensors  are  involved,  realistic  reproduc¬ 
tion  of  light  with  a  Sun-like  irradiation  intensity  and 
a  spectrum  is  crucial.  For  example,  during  VIBANASS 
open-loop  tests  (cf.  Section  2.2)  realistic  illumination 
conditions  including  different  incident  angles  had  to  be 
realized.  For  that  purpose,  a  spotlight  with  a  spectrum  very 
close  to  the  Sun  is  used  at  EPOS  [10]. 

2.2.  VIBANASS  test  campaign  on  EPOS 

In  the  VIBANASS  project  an  optical  rendezvous  and 
docking  camera  system  was  developed  [19,21,23].  The 
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Fig.  1.  Layout  of  EPOS  facility  -  two  industrial  robots  simulating  the  last 
25  m  of  the  rendezvous  phase  and  a  PC  based  monitoring  and  control 
system. 


requirements  were  based  on  low  Earth  orbit  and  geosta¬ 
tionary  on-orbit  servicing  scenarios  like  DEOS  (DEutsche 
Orbitale  Servicing  Mission  -  German  Orbital  Servicing 
Mission)  and  OLEV  (Orbital  Life  Extension  Vehicle). 

VIBANASS  consists  of  three  cameras  using  different 
optics  intended  for  far,  mid  and  close  range  rendezvous. 
All  three  sensors  are  based  on  the  same  CMOS  chip  which 
captures  10  bit  gray  scale  images  with  a  resolution  of 
1024  x  1024  pixels.  The  camera  system  further  comprises 
all  necessary  control  and  power  supply  hardware  and 
image  processing  software  like  JPEG  compression  and 
optical  correction.  The  VIBANASS  system  can  be  configured 
for  mono  and  stereo  image  acquisition.  Stereo  images  are 
mainly  used  for  close  range  rendezvous.  VIBANASS  further 
contains  a  laser-based  target  illumination  system  which 
improves  the  quality  of  images  captured  by  the  close  range 
camera.  A  detailed  technical  description  of  the  camera  and 
of  the  target  illumination  system  is  given  in  [19],  An  image 
processing  system  [24]  is  used  for  VIBANASS  which  con¬ 
tinuously  tracks  the  target  from  camera  images  and  which 
computes  an  estimation  of  the  3D  relative  position  of  the 
target  with  respect  to  the  camera. 

Tests  and  verification  of  the  mid  and  close  range 
cameras  were  conducted  at  EPOS  2.0  [21,23],  The  VIBANASS 
demonstrator  model  contained  a  camera  system  with  a 
close  range  stereo  camera,  a  mid  range  mono  camera  and  a 
target  illumination  system.  The  sun  simulator  of  the  EPOS 
facility  was  used  to  simulate  different  incident  angles  and 
thus  different  illumination  conditions.  The  robot  on  the 
linear  slide  carried  the  VIBANASS  sensor  system  and  acted 
as  a  service  satellite,  whereas  the  second  robot  carried  a 
mock-up  of  a  typical  geostationary  satellite,  see  Fig.  2. 

In  the  VIBANASS  test  campaign,  open-loop  tests  were 
performed  where  the  camera  system  and  the  image  proces¬ 
sing  system  were  tested  and  verified  under  different  illumina¬ 
tion  conditions  (incident  angle  of  the  Sun  simulator, 
brightness  of  the  light),  different  trajectories  and  different 
camera  settings  (e.g.  different  image  compression).  GPS  time 
was  used  for  time  synchronization  between  the  sensor  system 


and  the  EPOS  facility  such  that  the  images  could  be  off-line 
post-processed  using  the  EPOS  log  data  as  reference  data. 

A  more  realistic  rendezvous  simulation  was  additionally 
conducted  by  embedding  the  camera  and  the  image  processing 
system  in  a  closed  loop  simulation.  In  the  framework  of  the 
closed-loop  rendezvous  simulation,  the  robots  did  not  move 
according  to  a  predefined  trajectory  but  represented  as  a 
realistic  movement  of  two  satellites  in  space  based  on  physical 
equations  of  motions.  Sensor  data  was  processed  by  the  image 
processing  unit  and  the  navigation  system  and  handled  over  to 
the  controller.  The  controller  output  was  fed  back  to  the 
actuator  and  the  satellite  simulator.  GPS  time  was  used  again 
to  provide  a  time  stamp  for  each  captured  image;  a  necessary 
information  to  compensate  for  time  delays,  see  Section  2.3.2. 

An  existing  closed  loop  rendezvous  simulation  concept 
[11]  was  adapted  and  modified  for  the  VIBANASS  setup.  The 
main  challenge  was  the  handling  of  measurements  with  delay 
which  can  occur  during  on-orbit  servicing  with  teleoperation. 

Fig.  3  shows  a  ground-based  rendezvous  and  docking 
scenario  involving  geostationary  satellites  (left)  and  the 
simulation  concept  at  EPOS  for  closed  loop  hardware-in- 
the-loop  rendezvous  tests  (right).  In  a  ground-based  on- 
orbit  servicing  mission,  images  of  the  rendezvous  cameras 
are  provided  via  telemetry  to  the  ground.  The  image 
processing,  which  provides  relative  position  measure¬ 
ments,  and  the  corresponding  guidance,  navigation  and 
control  algorithms  are  computed  on  ground  and  are 
supervised  by  an  operator.  The  resulting  telecommands 
e.g.  the  actuators  of  the  servicer  or  commands  for  the 
sensor  control  are  sent  to  the  satellite.  At  EPOS,  the  space 
segment  is  simulated  by  the  manipulators  which  are 
controlled  in  real-time.  Their  motion  is  computed  by  a 
dynamical  satellite  simulator.  Furthermore,  interfaces  to 
the  cameras  are  established  for  control  of  the  sensors  and 
acquisition  of  image  data.  The  payload  control  system 
contains  the  image  processing  system  and  the  GNC  system 
for  the  control  of  the  relative  position. 

2.3.  Numerical  simulator 

The  numerical  simulator  contains  all  elements  which  are 
not  represented  by  real  hardware.  These  comprise  dynamical 
and  kinematical  models  for  the  position  and  orientation  of  the 
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Manipulators^  ^  ^ 


Fig.  4.  Chaser  spacecraft  simulator  for  VIBANASS. 


target  and  the  chaser  spacecraft,  actuator  models  and  the 
guidance,  navigation  and  control  system.  Further,  those  sensors 
which  are  not  available  as  real  hardware  in  the  VIBANASS  setup 
have  to  be  simulated  and  are  part  of  the  numerical  simulator. 

Fig.  4  gives  an  overview  on  the  numerical  simulator  for 
the  chaser  spacecraft.  The  images  captured  by  the  VIBA¬ 
NASS  camera  system  are  processed  by  the  image  proces¬ 
sing  unit,  see  [24]  for  a  detailed  description  of  the  target 
tracking  algorithm.  The  input  to  the  navigation  system  is 
the  measured  target  position  with  respect  to  the  camera 
and  the  time  of  the  image  capturing.  Based  on  the  position 
measurements,  a  navigation  filter  determines  an  estima¬ 
tion  of  the  chaser's  relative  position  with  respect  to  the 
target.  A  guidance  system  delivers  the  desired  position  of 
the  chaser.  The  orbit  controller  compares  the  estimated 
relative  chaser  position  with  the  desired  relative  position 
and  computes  the  necessary  commands  for  the  actuators. 


Actuator  models  transform  the  commands  to  forces. 
These  are  passed  to  the  orbit  dynamics  and  kinematics 
models  which  realize  equations  of  motion  for  the  orbit 
of  the  chaser.  The  solution  of  the  equations,  i.e.  the 
chaser's  position,  is  commanded  in  real-time  to  the  EPOS 
facility.  Similarly,  equations  of  motion  for  the  target's 
position  are  solved  and  commands  are  sent  to  the 
facility. 

The  EPOS  manipulators  move  according  to  the  com¬ 
manded  values  and  thus  stimulate  the  VIBANASS  camera 
sensor  system.  The  EPOS  testbed  requires  a  commanding  rate 
of  250  Hz.  Therefore  the  orbit  dynamics  and  kinematics  are 
executed  every  4  ms,  whereas  the  GNC  system  and  the 
actuator  models  run  every  100  ms. 

Among  the  systems  visualized  by  Fig.  4,  the  guidance, 
the  image  processing,  the  navigation  filter  and  the  orbit 
controller  are  executed  on-ground  (cf.  Fig.  3). 
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The  attitude  control  system  is  part  of  the  space  seg¬ 
ment  and  thus  not  affected  by  any  delay.  At  EPOS,  the 
dynamical  motion  simulator  (cf.  Fig.  3,  right  sub-figure) 
has  to  generate  both  position  and  attitude  commands  for 
the  6  DOF  manipulators.  Therefore,  a  simple  attitude 
system  is  realized  in  the  numerical  simulator.  The  attitude 
dynamics  and  kinematics  are  based  on  the  Euler  equation 
and  the  quaternion  differential  equation,  cf.  [25], 

The  following  sections  describe  the  single  elements  of 
the  closed  loop  rendezvous  simulation  in  detail. 


2.3.1.  Orbit  dynamics  and  kinematics 

The  objective  is  to  develop  a  realistic  simulation  of  the 
rendezvous  process  including  the  real  orbit  mechanics. 
A  numerical  model  based  on  equations  of  motion  is 
implemented  to  emulate  the  realistic  motion  of  the  two 
satellites  in  orbit.  The  position  and  the  velocity  are 
described  in  the  Clohessy  Wiltshire  (CLW)  coordinate 
framework,  see  [26],  It  is  a  local  coordinate  system  whose 
origin  is  aligned  with  the  center  of  mass  of  the  target 
spacecraft.  Thus,  the  chaser's  position  is  described  in  the 
local  orbital  frame  of  the  target.  Table  1  explains  the  axes 
of  the  CLW  coordinate  system. 

The  Hill  equations  describe  the  relative  motion  of  the 
active  or  service  spacecraft  with  respect  to  the  target 
spacecraft  on  a  circular  orbit  with  orbit  rate  ®0-  The  Hill 
equations  are  a  system  of  linear,  ordinary  differential 
equations  for  the  position  pt  =  (px,py,pz)  and  the  velocity 
"v"  =  (px,py,pz)  =  (yx,vy,vz)  of  the  service  satellite  in  the 
CLW  framework,  cf.  [26]: 

px  =  2(o0pz+^ifx,  (la) 


Py  =  -®oPy+~/y>  (lb) 

pz  =  -2ffl0px+3ffloPz+i/z.  (lc) 


Here,  m  is  the  mass  of  the  satellite,  f  =(fx,fy,fz)  is  the 
sum  of  external  forces  acting  on  the  satellite,  u  denotes  the 
derivative  with  respect  to  time  of  a  time-dependent 
function  u,  ii  ia  the  second  time-derivative  of  u. 

Given  the  initial  conditions,  i.e.  position  and  velocity  at 
time  t=  0,  the  initial  value  problem  can  be  solved  with 
standard  numerical  methods  like  Runge-Kutta  methods. 
In  practice,  for  simulation  on  EPOS  we  use  the  simple 
explicit  Euler  scheme  as  a  numerical  solver  since  this  is 
prescribed  by  the  real-time  configuration  at  EPOS. 


2.3.2.  Navigation  filter  with  delay 

The  navigation  filter  gives  an  estimate  of  the  relative 
position  and  a  velocity  of  the  service  satellite  with  respect 


to  the  target  based  on  prior  knowledge  of  the  state 
combined  with  measurements  observed  over  time. 

We  first  consider  a  discrete,  linear  system  with  non- 
delayed  measurements  and  additive  Gaussian  noise 
^k  1*1  —Fkx'k  +  Gfc  i/fc-h  2k,  ltk  ~  N(0,  Qk),  (2a) 

l k  —  N k  *  k  ^~  a* k-  k  N(0.  Rk)-  (2b) 

where  ltk  e  Rn  denotes  the  state  vector,  tk  e  Rl  the  input  or 
control  vector,  yk  e  R"  the  system  noise  with  zero  mean 
and  covariance  Q.k  e  Rnxn,  ~tk  e  denotes  the  measure¬ 
ment  and  vk  e  Rm  the  measurement  noise  with  zero  mean 
and  covariance  Rk  e  Rmxm.  Further  Fk  e  Rnxn,  Gk  e  Rnx', 
Hk  e  Rmxn  are  matrices  describing  the  linear  relation 
between  7fc+1  and  itk  and  tk  as  well  as  the  linear  relation 
between  ~?k  and  itk. 

The  Kalman  filter  [27]  algorithm  states  a  recursive 
solution  to  the  filtering  problem  (2).  It  consists  of  predictor 
and  corrector  steps  to  determine  an  estimation  "x  k  of  the 
state  and  an  estimation  P“l  of  the  covariance  of  itk—xk  . 
The  algorithm  is  given  by 

=  Fkltl*  +  Gktk,  (3a) 

Pk^=FkPTF'k  +  (lk,  (3b) 


U  PZ ,  H’k+ ,  (Hk+ ,  PP- ,  H[+ ,  +  + , )  - 1 ,  (3c) 

1 = +Kfc+i(^k+i -Hfc+1iO,  (3d) 

P?U=V-Kk+lHk+,)Pl“v  (3e) 

see  [27,28]  for  reference. 

Now,  we  consider  a  measurement  ~zk  which  is  delayed 
by  N  samples.  The  corresponding  measurement  equation 
can  be  formulated  as 

tk  =  Hsits  +  s|  mk—N.  (4) 

Thus,  z  k  is  the  measurement  of  the  state  it s  at  time  ts, 
s=k-N.  For  the  filter,  the  measurement  is  available  at 
time  tk. 

More  generally,  the  time  when  the  measurements  are 
performed  need  not  be  synchronized  with  the  discrete 
time  steps  when  the  filter  is  executed.  We  therefore 
consider 

tDk = t  ( t ) = H(t)t  (t) +a(t),  (5) 

where  t  <  tk.  The  delayed  filter  problem  is  illustrated  in 
Fig.  5. 

At  time  tfc,  we  face  the  following  delay  problem:  the 
measurement  tk  =  ~z(tk)  is  not  available  yet.  The  most 
recent  available  measurement  is  ~z  (t),  t  <  tk.  A  simple  and 
intuitive  way  of  adapting  the  Kalman  filter  algorithm  for 


Table  1 

Clohessy  Wiltshire  coordinate  framework. 


Axis  Axis  name  Description 


x  V-Bar  Tangential  direction,  i.e.  direction  of  the  orbital  velocity  vector 

y  H-Bar  Opposite  direction  of  the  angular  momentum  vector  of  the  orbit,  i.e.  parallel  to  the  normal  vector  of  the  orbit  plane 

z  R-Bar  Direction  to  Earth,  i.e.  radial  from  the  spacecraft's  center  of  mass  to  the  center  of  the  Earth 


H.  Benninghoff  et  al.  /  Acta  Astronautica  102  (2014)  67-80 


m 


Fig.  5.  Delayed  filter  problem. 


delayed  measurements  is  to  replace  the  measurement 
residuum  ~?k+i  -Hfc+1  x  k+1  in  (3d)  by 

At°:  =  ?(t)-H(t)lt(t)est,  (6) 

where  lt(tf%t  is  an  estimation  of  the  state  at  time  t. 
Therefore,  let  te[ts_i,ts],  for  some  s<k.  We  interpolate 
to  get  an  estimation  of  the  state  at  time  t: 


The  update  equation  (3d)  is  slightly  changed  to 


"^fc+1  =^k+l+Kk+l&Z  k+1.  (8) 

The  setting  of  the  measurement  residuum  (6)  can  be 
mathematically  justified  by  the  following  considerations 
which  are  based  on  linear  extrapolation  and  first  order 
Taylor  series  approximation:  the  basic  idea  is  to  extra¬ 
polate  the  measurement  by  setting 

:  =  ~z  (t)  +  H^r  -  H(t)t (t)est.  (9) 

Eq.  (9)  can  be  motivated  by  considering  the  measurement 
equation  and  a  Taylor  series  expansion  of  (t+ At)  around 
twith  At:  =  tk-t>  0: 

~Z  (t+ At)  =  H(t+At)lt (t+ At)  +  lt(t+  At) 

=  H(t+At)lt(t+At)+G)(t) 

+  At^(t)  +  0(At2) 

=  ~t  (t)  +  H(t  +  At)t  (t  +  At)  -  H(t)3?  (t) 

+  At^(t)  +  0(At2).  (10) 

The  state  lt(t)  at  time  t  is  replaced  by  the  estimation 
~x  (t)est  defined  in  (7).  The  state  if  (t+ At)  at  time  t+At  =  tk 
is  replaced  by  the  best  available  estimation  it  k  .  The 
terms  Afw  (t)+0(At2)  are  omitted  in  the  estimation. 
Therefore,  the  expression  (10)  is  approximated  by  (9). 

The  measurement  residuum  is  defined  as 

A  z'°  =  ?“tra-Hklffe,  (11) 

analogous  to  the  standard  Kalman  filter,  where  the  mea¬ 
surement  is  now  replaced  by  the  extrapolated  measure¬ 
ment.  If  Eq.  (9)  is  inserted,  we  finally  obtain  (6).  Thus  we 
showed,  that  the  simple  definition  (6)  for  the  measure¬ 
ment  residuum  is  sufficient  for  delayed,  linear  filter 
problems.  In  Section  3,  we  demonstrate  that  this  approach 
results  in  a  navigation  filter  which  is  stable  within  a 


hardware-in-the-loop  test  with  real,  delayed  sensor 
measurements. 

In  order  to  compute  it (t)est,  the  values  s  must 

be  stored.  If  the  measurement  is  delayed  by  a  maximum  of 
N  samples,  the  last  N  estimates  need  to  be  stored  for  future 
use.  Thus,  by  adapting  the  Kalman  filter  slightly  using  (6) 
and  (8),  we  result  in  a  simple  filter  technique  for  linear 
filter  problems  which  can  handle  delayed  measurements. 

We  now  state  how  the  above  presented  filter  can  be 
applied  for  rendezvous:  the  Hill  equations  (1)  are  discre¬ 
tized  with  respect  to  time  by  using  the  Euler  method  and 
are  rewritten  to  the  form  (2a).  Therefore,  let  tk  = 
(fk<  ^k)  e  K6  be  the  state  vector  and  i?fc  =  (1  /m)Tk  the 
input  vector.  As  above,  the  subscript  k  denotes  that  a 
function  is  evaluated  at  time  tk,  e.g.  ~frk  :  =  jf  (tk). 

The  matrices  Fk  and  Ck  in  the  system  model  (2a)  are  set 
to 


1  0  0  \ 

0  1  0 

0  0  1 

0  0  2a>o 

0  0  0 

—  2a>o  0  0 


Ck 


/0  0  0\ 
0  0  0 
0  0  0 
1  0  0 
0  1  0 
Vo  o  1/ 


(12) 


(13) 


where  Id6  denotes  the  6x6  identity  matrix  and 
Stk  :  =  tk+ ,  -  tk.  The  output  of  the  measurement  system 
~ik  is  a  measurement  of  the  position  ~^k.  Therefore  the 
matrix  Hk  in  (2b)  is  set  to 


/I  0  0  0  0  0\ 

Hk  =  0  1  0  0  0  0  .  (14) 

\0  0  1  0  0  oj 


2.3.3.  Guidance  and  control 

The  guidance  system  provides  reference  values  and 
generates  a  position  profile.  The  objective  of  guidance  is 
to  define  and  force  a  state  that  the  spacecraft  should  finally 
reach.  For  VIBANASS,  a  guidance  trajectory  has  been 
chosen  which  consists  of  (i)  a  hold  point  at  an  initial 
position,  e.g.  at  18  m  distance,  (ii)  a  smooth  approach  to 
5  m  with  a  maximum  velocity  of  5  cm/s,  (iii)  a  second  hold 
point  at  5  m,  (iv)  a  smooth  approach  to  1.8  m  with  a 
decreased  velocity  of  5  mm/s  and  (v)  a  final  hold  point  at 
1.8  m.  At  the  intermediate  hold  point  at  5  m  distance  to 
the  target,  a  switch  of  the  camera  system  from  mono  mid 
range  to  stereo  close  range  camera  takes  place. 

A  PID  controller  compares  the  reference  values  pro¬ 
vided  by  the  guidance  system  with  the  actual  values.  The 
result  of  the  navigation  filter,  the  estimated  position  and 
velocity,  serves  as  an  input  for  the  orbit  controller.  Let  e(t) 
be  the  position  error,  i.e.  the  difference  between  estimated 


H.  Benninghoffet  al.  /  Acta  Astronautica  102  (2014)  67-80 


position  and  guidance  value  and  e(t)  the  velocity  error,  i.e. 
the  difference  between  estimated  velocity  and  guidance 
velocity.  The  controller  output  u(t)  is  a  weighted  sum  of 
e(t),  its  integral  and  the  velocity  error  e(t): 

u(t)  =  kP  e(t)+ki  e(r)dr  +  kDe(t),  (15) 

where  kP,  k,  and  kD  are  called  proportional  gain,  integral 
gain  and  derivative  gain  respectively. 

By  using  the  well-known  PID  control  theory  [29,26]  an 
initial  guess  of  these  parameters  has  been  calculated  based 
on  the  desired  damping  behavior  and  negligence  of  cross 
couplings  between  the  different  components  (see  (1)). 
After  system  integration  a  final  tuning  has  been  performed 
by  small  adaptations  of  the  control  parameters. 

The  controller  computes  force  commands  for  the 
thrusters.  In  the  simulation,  actuator  systems  cannot  be 
represented  by  real-hardware  and  are  consequently  part  of 
the  numerical  simulator.  The  simplified  actuator  system 


implemented  for  VIBANASS  only  limits  the  forces  due  to 
the  specified  maximum  allowed  thrust  level. 

3.  Results 

Several  closed  loop  rendezvous  tests  were  performed  in 
the  framework  of  the  VIBANASS  test  campaign  at  the  HiL 
simulator  EPOS.  Due  to  simple  simulation  architecture  and 
simple  interface  structures  we  faced  a  time  delay  of  2-3  s 
(time-varying,  about  0.5  s  standard  derivation).  These 
delays  are  representative  for  delays  occurring  during 
teleoperation  [3,7,8],  Hence,  the  behavior  of  the  GNC 
system  under  delay  conditions  similar  to  a  real,  ground- 
based  rendezvous  could  be  tested. 

3.J.  Hold  point  at  5  m 

The  first  test  was  a  closed  loop  rendezvous  simulation  at 
a  hold  point  at  5  m  using  the  mono,  mid  range  camera.  The 
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image  processing  result  served  as  an  input  for  the  navigation 
filter.  Fig.  6  shows  the  measurement,  filter  estimate  and  real 
value  for  the  three  components  of  the  position.  The  closed 
loop  was  started  after  120  s,  i.e.  the  first  controller  commands 
were  computed  based  on  the  filter  estimates  after  120  s.  The 
estimation  error  was  approx.  5  cm  in  x-,  approx.  3.8  cm  in  y- 
and  approx.  1.5  cm  in  z-direction.  After  the  closed  loop  was 
started,  the  filter  estimates  were  fed  back  to  the  controller.  As 
the  guidance  values  were  5  m  for  x  and  0  m  for  both  y  and  z, 
the  satellite  in  the  simulation  moved  such  that  the  filter 
estimates  matched  with  the  guidance  values.  Therefore,  the 
systematic  measurement  errors  (measurement  biases) 
induced  an  error  in  the  real  position  compared  to  the 
guidance  values. 


3.2.  Approach  with  image  processing 

An  approach  from  18  m  to  approximately  2  m  was 
performed  using  the  image  processing  result  as  an  input 
for  the  navigation  system.  The  guidance  trajectory  was 
a  continuous  approach  from  18  m  to  1.8  m  with  a  hold 
point  at  5  m  as  described  in  Section  2.3.3. 

Fig.  7  shows  the  measurement,  the  filter  estimates  and 
the  real  position  of  the  chaser  (x-component).  The 
approach  took  place  from  the  x-direction  in  the  Clohessy 
Wiltshire  frame  (V-Bar),  thus  x  corresponds  to  the  distance 
between  chaser  and  target. 

Fig.  8  presents  the  error  of  the  measurement  and  of  the 
filter.  The  x-component  (distance)  was  most  difficult  to 
measure  from  mono  camera  images.  Therefore,  the  x- 


measurement  was  less  accurate  than  the  y-  and  z- 
measurement. 

After  approximately  80  s  the  loop  was  closed,  i.e.  the 
filter  estimates  were  fed  to  the  controller  and  the  first 
controller  commands  were  computed.  Due  to  measurement 
errors  which  did  not  have  zero  mean,  the  controller  used 
erroneous  values  for  the  actual  position  leading  to  perfor¬ 
mance  errors,  similar  to  described  in  Section  3.1.  At  5  m 
(after  ca.  900  s),  the  hold  point  was  reached  and  a  switch  to 
stereo  camera  was  accomplished.  The  measurement  error 
was  much  smaller  after  the  switchover  to  stereo  camera, 
which  resulted  in  a  better  performance.  This  can  clearly  be 
seen  considering  for  example  the  y-coordinate  in  Fig.  8. 

Fig.  9  shows  the  average  measurement  error  with  respect 
to  the  distance.  At  the  start  point  at  18  m,  there  was  a 
measurement  error  of  30-35  cm  in  the  distance  (cf.  x- 
component).  A  smaller  distance  compared  to  the  real  dis¬ 
tance  was  measured  (negative  error).  Caused  by  the  delay  of 
the  measurement,  a  positive  distance  error  was  noted  for 
distances  <  14  m  (=a  too  large  distance  measurement).  At 
about  15  m  the  negative  error  of  the  measurements  and  the 
positive  error  induced  by  the  delay  canceled  each  other. 

At  approximately  10  m,  there  was  a  velocity  of  5  cm/s, 
cf.  panel  4  in  Fig.  9  which  shows  the  average  velocity  in  the 
x-direction  with  respect  to  the  distance.  Having  a  time 
delay  of  2-3  s,  a  distance  error  of  10-15  cm  had  been 
expected.  This  was  consistent  with  the  experimental 
results  at  10  m,  see  panel  1  of  Fig.  9.  During  the  accelera¬ 
tion  and  deceleration  phase,  the  error  caused  by  the  delay 
was  smaller  since  the  velocity  was  smaller.  This  resulted  in 
a  local  maximum  of  the  error  in  the  x-coordinate  at  10-11  m. 


Measurement,  Estimates  and  Real  Position  Servicer 


Time  [s] 

Fig.  7.  Approach  18-2  m;  absolute  values  of  measurement,  filter  and  real  position,  x-component.  (For  interpretation  of  the  references  to  color  in  this  figure 
caption,  the  reader  is  referred  to  the  web  version  of  this  paper.) 
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At  distances  between  14  m  and  18  m,  the  pure  measure¬ 
ment  error  dominated  the  delay  error  which  resulted  in  a 
negative  total  distance  error. 

At  the  second  hold  point  at  5  m,  the  measurement 
improved.  The  delay  had  smaller  effect  since  the  velocity 
was  almost  0.  When  starting  the  second  approach  phase 
with  the  stereo  camera,  the  error  slightly  increased  as 
delays  resulted  in  a  positive  error.  However  during  the 
whole  approach  from  18  m  to  2  m,  a  decrease  of  the 
absolute  value  of  the  error  could  be  noted.  With  smaller 
distance  to  the  target,  the  accuracy  could  be  improved. 

3.3.  Approach  with  operator  in  the  loop 

An  approach  from  18  m  to  2  m  was  performed  with  an 
operator  in  the  loop.  The  same  guidance  trajectory  as  in 
Section  3.2  was  used.  The  operator  used  a  space  mouse  and 
marked  the  outer  edges  of  the  target  body  in  the  camera 
images  during  mid-range  and  the  boundary  of  the  nozzle 
during  close  range  rendezvous,  see  Fig.  10.  From  the  detected 


edges  the  relative  position  could  be  computed.  The  operator 
therefore  replaced  the  first  part  of  the  image  processing:  the 
detection  and  localization  of  edges  in  the  camera  image. 

Figs.  11-13  show  the  results  of  this  test,  in  detail  the 
absolute,  filter  and  measured  x-value,  the  error  of  filter 
estimates  and  measurements  for  all  components  and  the 
measurement  error  depending  on  the  distance.  Measure¬ 
ment  errors  and  little  measurement  rate  caused  the 
system  to  oscillate  at  the  initial  position  at  18  m.  But  the 
amplitude  of  the  oscillations  was  decreasing.  Thus  the 
system  was  still  stable.  When  the  distance  became  smaller 
during  the  approach,  the  measurements  became  more 
accurate.  For  distances  <  5  m,  the  measurement  error 
decreased  significantly  due  to  the  switch  from  mid  range 
to  close  range  camera. 

4.  Discussion 

The  results  of  the  first  test  at  5  m  hold  point  show  the 
general  behavior  of  the  GNC  system  if  the  measurement 
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distance  [m] 


Fig.  9.  Approach  18-2  m:  panels  1-3:  measurement  error  with  respect  to  distance,  blue:  mid  range,  cyan:  close  range  camera,  panel  4:  average  velocity 
(x-component)  w.r.t.  distance.  (For  interpretation  of  the  references  to  color  in  this  figure  caption,  the  reader  is  referred  to  the  web  version  of  this  paper.) 


Fig.  10.  Image  processing  operator  during  VIBANASS  test  campaign- 
manual  detection  of  the  client  in  camera  images  using  a  space  mouse. 


output  has  a  systematic  error.  To  transform  measurements 
from  a  camera  coordinate  system  to  world  coordinates 
a  hand-eye  calibration  is  needed  [30],  Hand-eye 


calibration  data  was  not  available  during  the  experiment. 
Therefore  the  main  source  of  measurement  errors  is 
missing  hand-eye  calibration.  These  biases  also  affect  the 
estimation  error,  since  the  filter  can  smooth  the  raw 
measurement  data,  but  cannot  correct  biases. 

The  systematic  measurement  errors  observed  during 
the  approach  (recall  e.g.  Fig.  8)  are  again  caused  by  missing 
calibration  but  also  by  the  time  delay.  The  delay  mainly 
affects  the  distance  component.  Further,  the  accuracy  of 
the  measurements  is  limited  by  the  resolution  of  the 
image:  at  large  distances,  changes  in  position  of  some 
centimeters  cause  only  sub-pixel  changes  in  the  position 
of  the  target  in  the  image.  Therefore,  these  changes  are 
hard  to  recognize  by  an  image  processing  system. 

The  measurement  error  in  x  at  zero  average  velocity 
(e.g.  at  hold  points)  decreases  with  decreasing  distance. 
The  measurement  error  induced  by  the  time  delay 
increases  with  increasing  velocity.  Although  delayed  mea¬ 
surements  are  extrapolated,  recall  Eq.  (9),  they  can  be  a 
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source  of  error  since  the  extrapolation  can  also  be  erro¬ 
neous.  For  example,  if  there  are  speed  changes,  the  position 
profile  will  be  non-linear  and  the  linear  extrapolation  of  the 
position  measurement  will  be  inaccurate.  At  all  distances 
however,  the  error  in  x  is  <  2%  of  the  distance  during  the 
test  where  the  image  processing  result  was  used  as  a 
measurement  input  for  the  navigation  system. 

In  the  y-  and  z-coordinate,  the  time  delay  has  no 
noticeable  effect  as  there  is  a  constant  guidance  value  in 
the  y-  and  z-direction  during  the  approach.  The  error 
decreases  with  decreasing  distance  to  the  target  since 
the  target  becomes  larger  in  the  camera  images.  Image 
processing  errors  e.g.  1  pixel  result  in  a  smaller  error  of  the 
final  position  measurement  compared  to  larger  distances. 
The  error  in  y  and  z  is  about  0.5%  of  the  distance  when 
using  the  mid  range  camera.  A  small  peak  at  11  m  (cf. 
Fig.  9)  seems  to  be  a  single  effect  and  could  be  caused  by 
problems  in  tracking  the  outer  edges  of  the  target.  Maybe 
there  was  some  edge  in  the  background  of  the  EPOS 
laboratory  near  the  target  edges  which  can  cause  problems 
in  image  processing  and  tracking. 

After  the  switchover  to  the  stereo  camera,  the  error 
significantly  improves.  At  a  close  range,  the  error  in  the 
y-  and  z-component  is  approximately  0.1 -0.2%  of  the 
distance. 

Compared  to  the  test  with  image  processing,  the 
approach  with  an  operator  in  the  loop  was  accomplished 
under  more  difficult  conditions: 

•  less  accurate  measurements:  the  target  is  hard  to  track 
accurately  by  the  operator,  in  particular  at  large  distances, 


•  reduced  measurement  rate:  0.2-0.3  Hz  average  rate, 
compared  to  the  image  processing  rate  of  1  Hz, 

•  less  continuous  measurements,  big  deviation  in  the  time 
between  two  measurements  (sometimes  2-3  s,  some¬ 
times  10  s  difference  between  two  measurements), 

•  positive  and  negative  measurement  errors  alternate. 


Despite  these  conditions,  a  stable  approach  could  be 
accomplished.  The  errors  and  the  oscillations  were  natu¬ 
rally  higher  compared  to  the  tests  with  image  processing. 

The  main  goal  of  the  closed  loop  rendezvous  simulation 
was  to  perform  a  safe  approach  from  about  18  m  to  a  point 
prior  to  docking.  The  closed  loop  proved  to  be  stable.  Small 
systematic  measurement  errors  and  varying  time  delays 
which  are  expected  to  occur  during  ground-based  on-orbit 
servicing  missions  could  be  handled.  Those  affected  the 
accuracy  and  the  performance  of  the  entire  system  but  the 
control  loop  kept  stable.  Since  time  delay  is  a  critical  issue  for 
the  stability  of  a  control  loop,  it  is  useful  to  work  on  methods 
that  compensate  for  time  delay.  If  the  approach  velocities  are 
small  and  the  delay  time  in  the  range  of  some  seconds,  time 
delays  can  be  handled  during  rendezvous  if  the  GNC  system 
is  accordingly  adapted  for  delayed  measurements. 

In  the  work  of  Bell  et  al.  [17],  time  delays  of  2.8  s  are 
observed  during  hardware-in-the-loops  tests  with  a  3D 
LIDAR.  The  delay  can  be  partly  compensated  by  a  Kalman 
filter  which  uses  a  guess  of  2.0  s  of  time  delay.  The  time 
difference  of  2.8  s  is  measured  after  the  experiments.  It  is 
not  described  if  and  how  the  standard  Extended  Kalman 
filter  is  adapted  for  time  delays.  In  our  experimental  setup, 
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we  are  able  to  measure  the  time  of  the  image  capturing 
using  a  GPS  receiver.  Further,  we  adapted  the  Kalman  filter 
slightly  to  handle  time  delays.  For  that  we  store  not  only 
the  actual  filter  estimate  but  a  certain  number  of  previous 
estimates  which  are  necessary  to  compute  an  estimate  of 
the  state  at  the  time  of  the  image  capturing,  recall  (7). 
In  addition,  there  is  no  restriction  on  the  time  delays  in  our 
setup.  The  delay  can  be  time-varying  (i.e.  non-constant) 
and  need  not  be  a  multiple  of  the  sample  time  of  the  filter. 

In  summary,  on-orbit  servicing  with  telepresence  is  an 
alternative  to  autonomous  servicing,  where  the  GNC 
system  is  executed  on  the  servicer’s  on-board  computer 
without  or  with  limited  operator  intervention.  The  tele¬ 
presence  servicing  method  seems  to  be  a  promising  option 
when  the  GNC  system  is  adapted  for  time  delays. 

5.  Conclusion 

We  presented  a  concept  for  closed  loop  rendezvous 
simulation  involving  optical  sensors.  The  GNC  system  could 
cope  with  delayed  measurements  and  enabled  a  stable 


approach.  Similar  time  delays  are  expected  for  future  servi¬ 
cing  missions  with  telepresence,  where  the  data  processing 
and  GNC  algorithms  will  be  computed  on  ground.  During  the 
VIBANASS  test  campaign  at  EPOS  2.0,  a  camera  system  and 
the  developed  GNC  system  were  embedded  into  a  rendez¬ 
vous  simulation  and  tested  in  a  closed  loop  mode.  Safe  and 
stable  approaches  from  18  m  to  2  m  with  two  camera 
systems  (mono  for  mid  range,  stereo  for  close  range)  were 
performed.  The  concept  was  able  to  use  both  image  proces¬ 
sing  results  and  measurements  by  an  operator.  Flowever,  the 
image  processing  yielded  more  accurate  measurements  and 
resulted  in  a  better  overall  performance  of  the  GNC  system. 
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